Papers with open-domain chatbots
Addressing Inquiries about History: An Efficient and Practical Framework for Evaluating Open-domain Chatbot Consistency (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to evaluate consistency capacity of open-domain chatbots are costly and low-efficient. |
| Approach: | They propose an efficient framework for evaluating consistency of open-domain chatbots . they use human judges to interact with chatbot, which is costly and low-efficient . |
| Outcome: | The proposed framework can assess the consistency capacity of chatbots and achieve a high ranking correlation with the human evaluation. |
MEDAL: A Framework for Benchmarking LLMs as Multilingual Open-Domain Dialogue Evaluators (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing meta-evaluation benchmarks are static, outdated, and lacking in multilingual coverage. |
| Approach: | They propose a framework for curating more representative open-domain dialogue evaluation benchmarks . they leverage several LLMs to generate user-chatbot multilingual dialogues conditioned on varied seed contexts based on a state-of-the-art LLM . |
| Outcome: | The proposed framework exploits state-of-the-art LLMs to perform multilingual evaluations of open-domain chatbots. |
ASEM: Enhancing Empathy in Chatbot through Attention-based Sentiment and Emotion Modeling (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models lack feature representations that capture the deep semantics of language and sensitivity to minor input variations, resulting in significant changes in the generated text. |
| Approach: | They propose an end-to-end model architecture called ASEM that performs emotion analysis on top of sentiment analysis for open-domain chatbots. |
| Outcome: | The proposed model outperforms existing models for generating empathetic embeddings, providing e-mpathetic and diverse responses. |
BotsTalk: Machine-sourced Framework for Automatic Curation of Large-scale Multi-skill Dialogue Datasets (2022.emnlp-main)
Copied to clipboard
| Challenge: | a number of largescale datasets targeting a specific conversational skill have recently become available. |
| Approach: | They propose a framework where multiple agents grounded to specific skills participate in a conversation to automatically annotate multi-skill dialogues. |
| Outcome: | The proposed framework can be used to build open-domain chatbots with diverse communicative skills. |
Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations (2023.emnlp-main)
Copied to clipboard
| Challenge: | open-domain chatbots focus on short single-session dialogue, neglecting the potential need for understanding contextual information in multiple consecutive sessions. |
| Approach: | They propose a 1M multi-session dialogue dataset for integrating time intervals and speaker relationships into a long-term conversation setup. |
| Outcome: | The proposed model can generate coherent responses according to time intervals and speaker relationships with high user engagement without contradiction in a long-term conversation setup. |